Network device driver architecture

ABSTRACT

The invention proposes a network device driver architecture with functionality distributed between kernel space and user space. The overall network device driver comprises a kernel-space device driver ( 10 ) and user-space device driver functionality ( 20 ). The kernel-space device driver ( 10 ) is adapted for enabling access to the user -space device driver functionality ( 20 ) via a kernel-space-user-space interface ( 15 ). The user-space device driver functionality ( 20 ) is adapted for enabling direct access between user space and the NIC ( 30 ) via a user-space-NIC interface ( 25 ), and also adapted for interconnecting the kernel-space-user-space interface ( 15 ) and the user-space-NIC interface ( 25 ) to provide integrated kernel -space access and user-space access to the NIC ( 30 ). The user-space device driver functionality ( 20 ) provides direct, zero-copy user-space access to the NIC, whereas information to be transferred between kernel space and the NIC will be “tunneled” through user space by combined use of the kernel-space device driver ( 10 ), the user-space device driver functionality ( 20 ) and the two associated interfaces ( 15,25 ).

TECHNICAL FIELD OF THE INVENTION

The present invention generally relates to a network device driverarchitecture for efficient and flexible access to a network interfacecontroller (NIC).

BACKGROUND OF THE INVENTION

Computer software can generally be divided into two types, operatingsystem software and application software. The operating system (OS) canbe viewed as a resource manager that makes the computer's resources suchas processors, memory, input/output (I/O) devices and communicationdevices available to the users. It also provides the base functionalityupon which application software can be written and executed. Importantoperating system functions include sharing hardware among users,preventing users from interfering with each other, resource scheduling,organizing data for secure and rapid access, and supporting I/Ofunctions and network communications.

The central part of the OS is commonly referred to as the kernel. Thekernel is normally only a portion of the code of what is commonlythought of as the entire OS, but it is one of the most intensively usedportions of the code. The kernel defines the so-called user-space, inwhich the application software runs, and provides services to userapplications, including memory management, allocating processingresources, and responding to system calls from user applications orprocesses. Other important kernel functions include interrupt handling,process management and synchronization, as well as I/O managementincluding network communications.

Since many different hardware devices can be connected to the computersystem, some of the I/O functionality is typically implemented as commonfunctionality that is device independent. Device related functionalityis then allocated within so-called device drivers. This means that auser application that needs to access a particular hardware device, suchas a network communication device, makes a system call to the OS, whichin turn invokes the device driver associated with the hardware device.

A Network Interface Controller (NIC) is a hardware device that iscommonly connected to computer systems for providing networkcommunication capabilities, such as Ethernet or ATM communication. NICcontrollers usually implement lower-level protocols, such as layer 1(PHY) and layer 2 (MAC, LLC) protocols, whereas higher level protocols(e.g. the TCP/IP protocol suite) traditionally are allocated in the OS,running in kernel mode. Moreover, clusters, for example, usually haveproprietary protocols running on top of Ethernet because TCP/IP(Transport Communication Protocol/Internet Protocol) is not very wellsuited for cluster computing in System Area Networks (SANs). Theseproprietary protocols are generally also running in kernel mode.

However, centralized in-kernel protocol processing prevents userapplications from realizing the potential raw performance offered by theunderlying high-speed networks. The performance problem is mainly causedby message copying between user space and kernel space, polluted cache,interrupts and non-optimized code. The intensive message copying createsa large overhead, especially for short messages, and constitutes themain reason for high processor load and low throughput of networksubsystems with standard operating systems.

This problem has become more pronounced with the advent ofhigh-performance network communication technologies such as GigabitEthernet, ATM and Infiniband. The main challenge in putting suchhigh-performance communication technologies into use lies primarily inbuilding systems that can efficiently interface these network media andsustain high bandwidth all the way between two network communicatingapplications.

This has lead the computer industry to develop network device driversthat support NIC access directly from user space, avoiding messagecopying between user space and kernel space. The most commonly knownexample of this type of user-space network access architecture is theVirtual Interface Architecture (VIA) developed by Intel Corporation,Microsoft Corporation and Compaq Computer Corporation. The VirtualInterface Architecture (VIA) is an industry standard for System AreaNetworks that supports direct, zero-copy user-space access to the NIC.The VIA Architecture was designed to eliminate message copying,per-message interrupts and other kernel overhead that have madetraditional networked applications become performance bottlenecks in thepast. As described, e.g. in the specification Intel Virtual Interface(VI) Architecture Developer's Guide, Sep. 9, 1998 and the InternationalPatent Application WO 00/41358, the VIA Architecture avoids intermediatedata copies and by-passes the operating system kernel to achieve lowlatency, high bandwidth communication. The VIA model includes a VIconsumer and a VI provider. The VI consumer typically includes a userapplication and an operating systems communication facility and a VIuser agent. The VI provider typically includes the combination of a VINIC and a VI kernel agent. The Virtual Interface (VI) is a directinterface between a VI NIC and a user application or process. The VIallows the NIC to directly access the user application's memory for datatransfer operations between the application and the network. The VIgenerally comprises a send queue and a receive queue, each of which canbe mapped directly to user address space, thus giving direct user-spaceaccess to the network level and by-passing the operating system kernel.

The technical report DART—A Low Overhead ATM Network Interface Chip,TR-96-18, July 1996 discloses an ATM NIC designed for high bandwidth,low overhead communication, by providing direct, protected applicationaccess to/from the network.

The main drawback of the VIA architecture (and similar architectures) isthat it requires special VIA-enabled NIC controllers, and can not run onoff-the-shelf NIC controllers such as ordinary Ethernet NIC controllers.Since a lot of functionality for network communication rely onkernel-level protocols such as TCP/IP, both a VIA-enabled NIC and anordinary Ethernet (TCP/IP) NIC are required with the VIA architecture.The VIA architecture is thus not optimized for implementation intoexisting systems, but generally requires hardware re-design of existingsystems, adding an extra NIC and/or NIC port to the system. Re-designinga circuit board, including design, testing, product handling,maintenance, spare parts, etc. may easily lead to extra costs in theorder of millions of dollars.

SUMMARY OF THE INVENTION

The present invention overcomes these and other drawbacks of the priorart arrangements.

It is a general object of the present invention to provide efficient andflexible access to a network interface controller (NIC), eliminating theCPU as the bottleneck in the communication chain.

It is also an object of the invention to provide an improved andcost-optimized network device driver architecture. In particular, it isbeneficial if the network device driver architecture is suitable forimplementation and integration into existing systems.

Yet another object of the invention is to provide a robust and flexiblenetwork device driver that is not NIC dependent and works with anyoff-the-shelf NIC hardware. These and other objects are met by theinvention as defined by the accompanying patent claims.

The general idea of invention is to provide an efficient, flexible andcost-effective network device driver architecture by means of integratedkernel-space access and user-space access to the NIC, preferably overthe same NIC port. This is accomplished by enabling direct user-spaceaccess to the NIC, in similarity to user-space network accessarchitectures, and most importantly enabling user-space tunneled accessbetween kernel-space and the NIC.

From an architectural point of view, the novel network device driverarchitecture normally comprises a kernel-space device driver as well asuser-space device driver functionality. The kernel-space device driveris adapted for enabling access between kernel space and user space via akernel-space-user-space interface. The user-space device driverfunctionality is adapted for enabling direct access between user spaceand said NIC via a user-space-NIC interface. This user-space devicedriver functionality is also adapted for interconnecting thekernel-space-user-space interface and the user-space-NIC interface toenable integrated kernel-space access and user-space access to the NIC.In this way, efficient user-space access to the NIC is obtained, whileat the same time kernel-level protocols are allowed to run over the sameNIC.

Preferably, the kernel-space device driver has two different operationalmodes. In the first mode, the kernel-space device driver is operable fordirectly accessing the NIC via a kernel-space-NIC interface. In thesecond mode, also referred to as user-space tunneled access mode, thekernel-space device driver is operable for accessing the NIC via theuser-space device driver functionality.

Advantageously, the user-space device driver functionality is configuredfor execution in application context of a user application, for exampleimplemented as user library functionality. For robustness and security,when the user-space tunneled access mode is activated, the operatingsystem orders the kernel-space device driver to switch back to the firstoperational mode if the user application crashes. As a second line ofdefense, or as an alternative, the kernel-space device driver mayoptionally be provided with a watchdog that switches back to the firstoperational mode if there has been no call from the user-space devicedriver functionality for a predetermined period of time.

In a preferred implementation, the kernel-space device driver has twobasic building blocks, the network device driver core and a kernel spaceagent. The network device driver core is preferably based on a standardnetwork device driver, for example obtained from a commercial vendor,with additional functionality for making the device driver work in bothdefault mode as well as the user-space tunneled access mode of theinvention. In default mode, the network device driver core operates asan ordinary network device driver, directly accessing the NIC. Inuser-space tunneled access mode, the driver core routes outgoing data tothe kernel agent and receives incoming data from the kernel agent. Thekernel agent manages the kernel-space-user-space interface, and supportstransfer of information to/from the user-space device driverfunctionality. The kernel agent generally comprises functionality commonto different types of NIC controllers, thus allowing easy adaptation ofstandard network device drivers for a particular NIC to the novelnetwork device driver architecture supporting user-space tunneled accessbetween kernel space and the NIC.

In conclusion, the invention allows simultaneous user-space andkernel-space access to the network layer over the same NIC port, thusleading to a reduction of the number of required NIC ports andeliminating the need for hardware re-design. By running on top of thesame NIC, smaller footprint/cost and better network utilization can beachieved. The novel network device driver architecture is well suitedfor applications that need high performance network communication aswell as functionality relying on kernel-level protocols. Examples ofsuch applications can be found in embedded environments, communicationsystems and so forth.

It should be understood that the expressions “NIC access” and “access tothe NIC” include both sending information to and receiving informationfrom the network level.

Other benefits of the novel network device driver architecture include:

-   -   Reduced hardware space and power dissipation, which is        especially important for embedded type of systems;    -   Less cabling;    -   Reduced number of ports required on the associated communication        switches, thus allowing the use of smaller and cheaper switches;        and    -   Efficient use of bandwidth in the network.

Further advantages offered by the present invention will be appreciatedupon reading of the below description of the embodiments of theinvention.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention, together with further objects and advantages thereof,will be best understood by reference to the following description takentogether with the accompanying drawings, in which:

FIG. 1 is a schematic general block diagram of a network device driverarchitecture according to a preferred embodiment of the invention;

FIG. 2 illustrates integrated user-space access and kernel-space accessto the NIC supported by zero-copy message transfer within the networkdevice driver according to the invention;

FIG. 3 is a schematic block diagram illustrating a preferred realizationof the network device driver architecture according to the invention;

FIG. 4 is a schematic flow diagram of a method for network accessaccording to a preferred embodiment of the invention;

FIGS. 5-10 are simplified views illustrating different traffic cases inthe distributed network device driver architecture of FIG. 3; and

FIG. 11 illustrates a particular example of an overall systemimplementation.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

Throughout the drawings, the same reference characters will be used forcorresponding or similar elements.

FIG. 1 is a schematic general block diagram of a network device driverarchitecture according to a preferred embodiment of the invention. Thenetwork device driver architecture is illustrated in its systemenvironment, including user space, kernel space as well as networkspace.

The invention proposes a network device driver architecture in which afraction of the standard device driver functionality is distributed touser space providing direct NIC communication, and the kernel-spacedevice driver has additional functionality for NIC access via userspace. The network device driver functionality is thus distributedbetween kernel space and user space, and the overall network devicedriver comprises a kernel-space device driver 10 and user-space devicedriver functionality 20. The kernel-space device driver 10 is adaptedfor enabling access to the user-space device driver functionality 20 viaa kernel-space-user-space interface 15. The user-space device driverfunctionality 20 is adapted for enabling direct access between userspace and the NIC 30 via a user-space-NIC interface 25, and also adaptedfor interconnecting the kernel-space-user-space interface 15 and theuser-space-NIC interface 25 to provide integrated kernel-space accessand user-space access to the NIC 30. The user-space device driverfunctionality 20 provides direct, zero-copy user-space access to theNIC, whereas information to be transferred between kernel space and theNIC will be “tunneled” through user space by combined use of thekernel-space device driver 10, the user-space device driverfunctionality 20 and the two associated interfaces 15, 25.

In this way, efficient user-space access to the NIC 30 is obtained,while at the same time kernel-level protocols 45 are allowed to run overthe same NIC. The network device driver architecture of the inventionsupports usage of a dedicated NIC port for user-space traffic to/from auser application 40, but also supports efficient sharing of the sameport for both kernel-level protocols and user-level protocols. Thepossibility of sharing the same NIC port generally opens up forcost-optimized solutions. Another important benefit of sharing the sameNIC port is the possibility to integrate the novel device driverarchitecture into existing systems without hardware modifications. Thus,system re-design may be avoided, leading to cost savings in the order ofseveral million dollars.

Preferably, the kernel-space device driver 10 has two differentoperational modes. In the first mode, the kernel-space device driver 10operates as a standard network device driver directly accessing the NIC30 via a kernel-space-NIC interface 35. In the second mode, alsoreferred to as user-space tunneled access mode, the kernel-space devicedriver 10 is operable for accessing the NIC 25 by means of theuser-space tunneling mechanism described above.

Advantageously, the user-space device driver functionality 20 isconfigured for execution in application context of a user application40, for example implemented as user library functionality. It isimportant that the kernel-level protocols 45 are not stalled in the caseof a user application crash or deadlock. In user-space tunneled accessmode, the operating system orders the kernel-space device driver 10 toswitch back to the first operational mode if the user applicationcrashes. The kernel-space device driver 10 now accesses the same NICport as the user application did before it crashed. As a second line ofdefense, or as an alternative, the kernel-space device 10 driver may beprovided with an optional software watchdog 12 that switches back to thefirst operational mode if there is no call from the user-space devicedriver functionality 20 for a predetermined period of time.Alternatively, a counter-based hardware watchdog can be connected to thenetwork device driver architecture.

In a preferred embodiment of the invention, all of the communicationinterfaces 15, 25 and 35 within the novel network device driverarchitecture support zero-copy transfer of information. For a betterunderstanding of the invention, an example of integrated user-spaceaccess and kernel-space access to the NIC supported by zero-copy messagetransfer within the network device driver will now be described withreference to FIG. 2. Each of the interfaces 15, 25 and 35 is preferablybased on a shared memory structure, for example in the form of bufferqueues. Each interface is normally associated with a send queue (KTX;TX; NTX) and a receive queue (KRX; RX; NRX). The buffer queues aretypically adapted for holding pointer information, and accessed bywriting for the tail and reading from the head. The pointer informationpoints to the real data such as a message stored in common memory.

The information transfer will now be described in the outbound directionfrom user application to NIC, both for user-level protocols as well asfor kernel-level protocols. It is apparent that the information transferis similar in the inbound direction.

In the case of a user-space terminated protocol, a message MSG-1 to besent from a user application 40 to the NIC 30 is stored in common systemmemory 50 or any other memory that can be accessed by the involvedsystem components. A pointer P-1 that points (dashed line) to thecorresponding memory position in system memory 50 is delivered to theuser-space device driver functionality 20 together with a request forNIC access. The user-space device driver functionality 20 puts thepointer into the TX queue (located in user address space) of theuser-space-NIC interface 25. The NIC 30 subsequently consumes themessage by reading the pointer from the TX queue and performing a directmemory access (DMA) from the corresponding position in the system memory50 to fetch the message.

In the case of a user application 40 in need of a kernel-level protocol,the user application makes a corresponding system call, and the messageto be transferred to the NIC 30 is copied into kernel-space and handledby the invoked kernel-space protocol 45. Once the message MSG-2 is inkernel-space, there will generally be no more message copying. Instead,the kernel-level protocol 45 delivers a pointer P-2 that points (dashedline) to the memory position of the message in system memory 50 to thekernel-space device driver 10, which inserts the pointer into the KTXqueue of the kernel-space-user-space interface 15. The user-space devicedriver functionality 20 polls the KTX queue and moves the pointer to theTX queue of the user-space-NIC interface 25. Once, the pointer has movedto the head of the queue, the NIC 30 will read the pointer and fetch thecorresponding message through a DMA access to system memory 50.

Preferably, all buffer queues are allocated in kernel address space bythe kernel-space device driver. The queues are mapped to the addressspace of the user-space device driver functionality. To make the queuesvisible to the NIC, they are first mapped to the NIC bus address spaceand the obtained addresses are then written to the specific NICregisters.

By working with message pointers, instead of complete messages, therewill be no actual message copying.

FIG. 3 is a schematic block diagram illustrating a preferred realizationof the network device driver architecture according to the invention.The kernel-space device driver 10 preferably has two basic buildingblocks, a network device driver core (NDD core) 14 and a kernel spaceagent 16. Together with the user-space device driver functionality 20,the NDD core 14 and the kernel agent 16 generally define the overallnetwork device driver architecture.

User-space messages are exchanged between user space and NIC withoutkernel involvement, and since the user-space device driver functionalitytypically works in polling mode, there will be no per messageinterrupts. Messages originating from kernel-level users are tunneledbetween the NDD core 14 and the NIC 30, via the kernel agent 16, theuser-space device driver functionality 20 and the associated interfaces15, 25.

Most operating systems such as Tru64, Linux, Windows and OSE supportsome form of device driver framework, which comprises a set of rules,interfaces and guidelines on how to develop device drivers. Theseframeworks are well documented and OS vendors often supply tools forgenerating device driver templates, thus saving valuable design time andeffort for developing new device drivers. The network device driver core14 as well as the kernel agent 16 are generally implemented according toa suitable device driver framework.

The network device driver core 14 is preferably based on a standardnetwork device driver, for example obtained from a commercial vendor,with additional functionality for making the device driver work in bothdefault mode as well as the user-space tunneled access mode of theinvention. Source code for the design base network device driver canusually be obtained from the device driver vendor, or by using freelyavailable source code (Linux, NetBSD and FreeBSD for example). Thedesign base adaptation for allowing user-space tunneling can typicallybe realized by adding about 50 lines of code (˜1% of the design basecode) to the design base device driver. It is also possible to designthe NDD core 14 in-house by using any of the available tools forgenerating device drivers.

In default mode, the NDD core 14 operates as an ordinary network devicedriver, directly accessing the NIC.

In user-space tunneled access mode, the NDD core 14 routes outgoing datato the kernel agent 16 and receives incoming data from the kernel agent.The NDD core or the user-space device driver functionality preferablyalso masks interrupts related to message processing since the user-spacedevice driver functionality 20 normally works in polling mode.

Conveniently, the kernel agent 16 performs some initializationprocedures, allocates contiguous memory, implements thekernel-space-user-space interface 15 as well as the interface to/fromthe NDD core 14, and maps contiguous memory and memory mappedconfiguration and state registers (CSR) to the address space of theuser-space device driver functionality 20. The kernel agent 16 supportstransfer of messages between the NDD core 14 and the user-space devicedriver functionality 20 via the kernel-space-user-space interface 15.Since the FIFO queues KTX, KRX of the kernel-space-user-space interfaceare allocated in kernel address space and mapped to user address space,no message copying is required between the kernel agent 16 and theuser-space device driver functionality 20. The kernel agent module isgenerally not dependent on the particular NIC used by the system, andcan transparently and simultaneously support different types of NICcontrollers, including Fast Ethernet, Gigabit Ethernet and ATM NICcontrollers.

The kernel agent 16 may also be adapted for monitoring the status of anyprocess using the user-space device driver functionality 20. This makesit possible for the kernel agent to order the NDD core 14 to switch backto default mode in the case of a user process failure.

In a typical case, the kernel agent 16 may be realized by approximately200 lines of new code together with about 300 lines of standard devicedriver framework code.

As mentioned above, the user-space device driver functionality 20 is asmall part of the overall device driver functionality, and preferablyimplemented as user library functionality executing in user space. Itnormally works in polling mode and supports direct exchange of messagesbetween user-space and NIC. Typically, the user-space device driverfunctionality may be realized by approximately 200 lines of code.

The interface between the kernel-level protocols 45 such as TCP/IP andDLI (Data Link Interface) on one hand and the NDD core 14 on the otherhand is conveniently an existing network device driver API (ApplicationProgramming Interface) supplied with the OS.

The interface between the NDD core 14 and the kernel agent 16 isnormally an API that supports sending/receiving messages over a specificNIC.

The interface 15 between the kernel agent 14 and the user-space devicedriver functionality 20 is preferably realized as a standard fileinterface, supporting user-space device driver functionality requestsfor opening a connection towards the kernel agent, mapping of contiguousbuffer memory and memory mapped CSR from the kernel agent to applicationcontext. If desired, it may also support the watchdog functionalityimplemented in the kernel agent as well as NIC status notification fromthe kernel agent 16 to the user-space device driver functionality 20.Message transfer between the kernel agent 14 and the user-space devicedriver functionality 20 is realized by means of a shared memorystructure, as previously described.

The interface between the user application 40 and the user-space devicedriver functionality 20 is normally an API that supportssending/receiving messages directly between the user address space andthe NIC 30, in combination with the FIFO-queue based interface 25between the user-space device driver functionality 20 and the NIC 30.This interface can be realized as a standard VI interface.

FIG. 4 is a flow diagram of a method for network access according to apreferred embodiment of the invention. In step S1, direct access betweenuser space and the NIC is provided via a user-space-NIC interface. Instep S2, which relates to the default operation mode, direct accessbetween kernel space and the NIC may be provided via a kernel-space-NICinterface. In user-space tunneled access mode, access between kernelspace and NIC is provided in two steps, by providing akernel-space-user-space-interface in step S3 and by interconnecting thekernel-space-user-space interface and the directuser-space-NIC-interface in step S4. The robustness and reliability ismaintained by switching back to default mode in step S5 if theapplication crashes. As a second line of defense, or as an alternativeto step S5, a watchdog may be used as in step S6, switching back todefault mode if there has been no call from user space for apredetermined period of time. In the following, illustrative examples ofa number of different traffic cases in the distributed network devicedriver architecture of FIG. 3 will be described with reference to FIGS.5-10, each of which only illustrates those system components that arerelevant to the respective traffic case.

The computer system boots up and loads the operating system. Theoperating system is loaded with a network device driver in kernel spacefor allowing access to the network and more particularly to the networkinterface controller (NIC) 30. The kernel-space device driver willperform a standard initialization procedure as a design base networkdevice driver (NDD), also referred to as the NDD core 14.

Additional functionality is registration to the kernel agent (not shown)from the attach procedure called for every NIC that the driver controls.In the registration procedure, a CSR pointer is sent to the kernelagent, and some callback functions are registered in the kernel agent(for switching between modes, resetting the NIC, pushing inboundmessages received through the kernel agent, error reporting, and soforth).

For every NIC controlled by the NDD driver, the operational mode is setto default, meaning that the message flow will go from the NDD core 14directly to the NIC 30 and vice versa, as illustrated in FIGS. 5 and 6.In the outbound direction, illustrated in FIG. 5, the NDD core 14operates as a design base NDD. Additional functionality involveschecking whether the user-space tunneled mode is activated for thisinterface. If the user-space tunneled access mode is off, messages tothe NIC are put in the NTX ring of the NDD core 14. In the inbounddirection, illustrated in FIG. 6, the NDD core 14 operates as a designbase NDD, and messages from the NIC are put in the NRX ring.

Next, a user application is initialized and started. The user-spacedevice driver functionality 20 executing in application context of theuser application opens a connection to the kernel agent 16 and indicateswith which port it wants to communicate. It also requests from thekernel agent 16 the mapping of DMA area and CSR registers to its ownaddress space, and the kernel agent 16 maps memory visible to the NIC 30to user address space. The user-space device driver functionality 20orders the kernel agent 16 to switch to user-space tunneled access mode.The kernel agent forwards the order to the NDD core 14, which firstperforms a reset and then marks that the user-space tunneled access modeis activated for this particular NIC interface. The NDD core 14 thenperforms basic initialization of the NIC 30, and the kernel agent 16preferably initiates the watchdog functionality.

The user-space device driver functionality 20 sets registers in the NIC30 indicating where the TX, RX rings are located. It may also set theNIC registers for improving performance over PCI (Peripheral ComponentInterconnect) or any other bus structure used by the invention.Subsequently, the user-space device driver functionality 20 enables theTX, RX engines.

In the outbound direction from user space directly to the NIC, asillustrated in FIG. 7, the user application delivers message descriptors(similar to the descriptors used in the VIA architecture) or simplepointers to the user-space device driver functionality 20, which putsthe message descriptors in the TX ring located in user address space. Byworking with message descriptors/pointers, instead of complete messages,there will be no actual message copying. The NIC 30 consumes themessages by using the message descriptors in the TX ring to find therelevant position in common memory.

In the outbound direction from a kernel-level user to the NIC, asillustrated in FIG. 8, the NDD core 14 operates as a design base NDDwith the additional functionality of checking if the user-space tunneledaccess mode is activated for this NIC interface. If the tunneled accessmode is activated, the NDD core 14 delivers incoming message descriptorsto the kernel agent 16, which puts the descriptors in the KTX ring usedfor communication with the user-space device driver functionality 20.

At a configurable interval, for example every 1 ms, the user-spacedevice driver functionality 20 will normally poll the KTX ring andmove-insert descriptors into the TX ring used for communication betweenthe user-space device driver functionality 20 and the NIC 30. The NIC 30finally consumes the messages by fetching them from common memory usingthe transferred message descriptors.

Two different traffic cases in the inbound direction from the NIC areillustrated in FIGS. 9 and 10. The NIC 30 puts descriptors associatedwith inbound messages in the RX ring, and the user-space device driverfunctionality 20 is polled for incoming message descriptors. The messagedescriptors are fetched from the RX ring. For each message, it isinvestigated whether the (Ethernet) type matches the one that the userapplication expects. If there is a match, the message is delivered tothe user, as illustrated in FIG. 9. If there is no match, the messagedescriptor is inserted into the KRX ring, corresponding to FIG. 10. At aconfigurable interval, for example every 1 ms, the user-space devicedriver functionality 20 calls the kernel agent 16. The kernel agent 16then fetches descriptors relating to inbound messages from the KRX ringand delivers them to the NDD core 14, which performs the necessaryoperations for delivering the messages to the kernel-level user.

If the user application crashes, the operating system will normallyinform the kernel agent 16, which in turn will order the NDD core 14 toswitch back to default mode. As an optional second line of defense, oras an alternative, the kernel agent 16 may be provided with a watchdogfor handling the case when the operating system for some reason fails toinform the kernel agent 16 that the user application has crashed. Forexample, the watchdog may be implemented in such a way that it ordersthe NDD core 14 to switch back to default mode if there is no call fromthe user-space device driver functionality 20 for a predeterminedperiod, for example 10 ms. This will make sure that the NDD basefunctionality is never lost for more than a predetermined minimum periodof time.

A particular example of an overall system implementation is illustratedin FIG. 11. The overall system comprises:

-   -   Two central processor base boards (CPBB-A, CPBB-B). Each CPBB        board comprises one or more Alpha servers (EV68ab, 667 MHz) and        1 GB of RAM.    -   Two updating bus boards (UPBB-A, UPBB-B), one per CPBB. Each        UPBB board comprises an Intel 82543GC Gigabit Ethernet NIC,        together with other UPBB components.    -   A 32 bit/66 MHz cPCI bus between CPU and the cPCI-PCI bridge,        and a 64 bit/33 MHz PCI bus between the cPCI-PCI bridge and the        Intel 82543GC NIC.    -   A Gigabit Ethernet fiber connection (SX) between the two        CPBB-UPBB pairs, which are connected back-to-back.    -   Two base I/O units, each of which is connected to the cPCI-PCI        bridge, and to an optional network hub as well as serially        connected to a PC-console.

In prototype testing, one CPBB was used for execution of the serverinstance of a test program, the other CPBB for execution of the clientinstance of the test program. The PC-console is used for accessing theapplication running on the CPBBs.

Measurements indicate that with efficient protocol termination in userspace it is actually possible to eliminate the CPU as a bottleneck inthe communication chain. In full-duplex tests, the PCI bus becomes thebottleneck, while in half-duplex tests for messages larger than 498octets, Ethernet becomes the bottleneck. The fact that the PCI busbecomes the bottleneck is not surprising since it is a shared bus.Today, there exist much better PCI busses on the market. This particularIntel NIC supports 64 bit/66 MHz PCI bus, substantially doubling thecapacity compared to using the 64 bit/33 MHz PCI bus mentioned above.The PCI-X also on the market today runs on 64 bit/133 MHz. The PCI-X 2.0will enable the bus to run in dual data rate (266 MHz) as well asquadruple data rate (532 MHz). In addition, PCI-SIG (PCI SpecialInterest Group) will release PCI 3.0, which is a switch to serialinterconnection enabling faster throughput, smaller footprint andbackward compatibility towards software users. Obviously, there is asmooth path in increasing PCI bus throughput in the system.

Measurements have shown that with the co-existence of user-space andkernel-space protocols over the same Ethernet port, it is possible toachieve a stable rate of 435 000 signal pairs per second (50 octets ofdata) between the two Alpha-based CPBB servers over PCI and GigabitEthernet with just 28.73% processor load. With a standard network devicedriver in the same processor environment, the processor load will reach100% already at 46 000 signal pairs per second.

The embodiments described above are merely given as examples, and itshould be understood that the present invention is not limited thereto.It is also important to note that the invention is not dependent oncommunication technology (Ethernet, ATM, Infiniband, . . . ), operatingsystem (Tru64, Linux, Solaris, Win, . . . ) or host CPU (Alpha, Intel,PowerPC, . . . ).

Furthermore, the invention is relevant to all types of information anddata, as well as all types of networks, including LAN (Local AreaNetwork), MAN (Metropolitan Area Network), WAN (Wide Area Network) andSAN (System Area Network) networks.

Further modifications, changes and improvements which retain the basicunderlying principles disclosed and claimed herein are within the scopeof the invention.

1. A method for enabling access between operating system kernel spaceand a network interface controller (NIC) as well as between user spaceand said NIC, wherein the method is performed by a kernel-space devicedriver and a user-space device driver, said method comprising the stepsof: enabling access between kernel space and user space via akernel-space-user-space interface; enabling direct access between userspace and said NIC via a user-space-NIC interface; interconnecting saidkernel-space-user-space interface and said user-space-NIC interface toenable user-space tunneled access between kernel-space and said NIC;providing simultaneous user-space and kernel-space access to a networklayer over a single NIC port; in a first operational mode of thekernel-space device driver, directly accessing the NIC from thekernel-space device driver via a kernel-space-NIC interface; and in asecond operational mode of the kernel-space device driver, accessing theNIC via the interconnected kernel-space-user-space interface anduser-space-NIC interface; wherein the step of enabling direct accessbetween user space and the NIC and the step of interconnecting thekernel-space-user-space interface and the user-space-NIC interface areexecuted in application context of a user application, and an operatingsystem orders the kernel-space device driver to switch to the firstoperational mode in response to a user application failure; wherein thekernel-space device driver switches to the first operational mode ifthere is no user-space call to the kernel-space device driver for apredetermined period of time.
 2. The method according to claim 1,wherein said interconnecting step comprises the steps of: fetchingpointer information, pointing to data in a common memory, from a memorybuffer associated with one of said kernel-space-user-space interface andsaid user-space-NIC interface; and inserting said pointer informationinto a memory buffer associated with another of said interfaces.
 3. Themethod according to claim 1, further comprising, for outboundkernel-level protocol communication, the steps of said kernel-spacedevice driver inserting pointer information, pointing to data in acommon memory, into a transmit buffer associated with saidkernel-space-user-space interface, and said user-space device driverfunctionality fetching said pointer information therefrom and insertingit into the transmit buffer associated with said user-space-NICinterface, and said NIC fetching said pointer information from thetransmit buffer associated with said user-space-NIC interface andreading corresponding data from said common memory based on the obtainedpointer information.
 4. The method according to claim 1, furthercomprising, for inbound kernel-level protocol communication, the stepsof: said NIC inserting pointer information, pointing to data in a commonmemory, into a receive buffer associated with said user-space-NICinterface; said user-space device driver functionality fetching saidpointer information from the receive buffer associated with saiduser-space-NIC interface and inserting it into the receive bufferassociated with said kernel-space-user-space interface; and saidkernel-space device driver fetching said pointer information fortransfer to a kernel-level protocol, which reads the corresponding datafrom said common memory based on the pointer information.
 5. The methodaccording to claim 1, wherein said step of enabling direct accessbetween user space and said NIC and said interconnecting step areexecuted in application context of a user application.
 6. The methodaccording to claim 5, wherein said step of enabling direct accessbetween user space and said NIC and said interconnecting step areperformed by user-space device driver functionality implemented asuser-space library functionality.